Despite the potential of reinforcement learning (RL) for building general-purpose robotic systems, training RL agents to solve robotics tasks still remains challenging due to the difficulty of exploration in purely continuous action spaces. Addressing this problem is an active area of research with the majority of focus on improving RL methods via better optimization or more efficient exploration. An alternate but important component to consider improving is the interface of the RL algorithm with the robot. In this work, we manually specify a library of robot action primitives (RAPS), parameterized with arguments that are learned by an RL policy. These parameterized primitives are expressive, simple to implement, enable efficient exploration and can be transferred across robots, tasks and environments. We perform a thorough empirical study across challenging tasks in three distinct domains with image input and a sparse terminal reward. We find that our simple change to the action interface substantially improves both the learning efficiency and task performance irrespective of the underlying RL algorithm, significantly outperforming prior methods which learn skills from offline expert data.

We present a transfer learning strategy which fundamentally relies on Bayesian deep learning and the ability to represent Exploration and adaptation to new tasks in a transfer a distribution over functions, as in (Bachman et al., 2018) learning setup is a central challenge in reinforcement (Garnelo et al., 2018). Bayesian methods rely on modeling learning. In this work, we build on the uncertainty over value functions to represent the agent's the idea of modeling a distribution over policies belief of the environment. Recent work has shown that in a Bayesian deep reinforcement learning setup neural networks can be used to represent an uncertainty to propose a transfer strategy. Recent works over the space of all possible functions (Bachman et al., have shown to induce diversity in the learned 2018). The idea of modeling a distribution over functions policies by maximizing the entropy of a distribution can be adapted in the RL setting to model a distribution over of policies (Bachman et al., 2018; Garnelo policies, such that we can also maximize the entropy over et al., 2018) and thus, we postulate that our this distribution of policies. This is similar to maximum proposed approach leads to faster exploration resulting entropy exploration in RL, where instead of local entropy in improved transfer learning. We support maximization, recent work maximizes the global entropy our hypothesis by demonstrating favorable over the space of all possible sub-optimal policies.